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ost students seeking to enroll in community college are required to take a placement test 

for math and English courses. Among those students, 60 percent are directed into at least 

one developmental (remedial) education course. Community colleges have often relied 
solely on a traditional placement test, such as College Board’s ACCUPLACER, but recent evidence 
suggests that this approach misplaces substantial numbers of students. While some students are 
misdirected into college-level courses they are not ready for, research indicates that up to one-third 
of those placed into developmental education could be successful in college-level courses. These 
underplaced students are investing time and money on courses that do not contribute to their degree 
plans, which may discourage them from continuing their education. 


Administrators are well aware of the shortcomings of using traditional placement tests alone, and 
state and college systems across the country are organizing multiple measures assessment (MMA) 
placement systems that will potentially provide more accurate results. MMA systems use alternative 
measures alongside the traditional tests to create a final placement. Among available measures, there 
is strong empirical evidence that high school grade point average (GPA) is one of the best predictors 
of college success. While the evidence is less strong for noncognitive assessments — measures of 
characteristics such as motivation and problem-solving skills — some research shows that they too 
can be predictive of success in college courses. 


During fall 2016, MDRC, in partnership with the Community College Research Center (CCRC), 
launched the Multiple Measures Assessment Project, funded by Great Lakes Higher Education 
Corporation, and assisted ten Minnesota and Wisconsin colleges with the design and piloting of 
MMA placement systems. In the pilot in summer 2017, students who were identified as needing 
developmental education based on test scores could be “bumped up” to a college-level course based 
on measures such as high school performance (primarily GPA) or a noncognitive assessment. 


This guide is a result of that project. While recent evidence shows that implementing an MMA sys- 
tem is a promising strategy to mitigate the issue of underplacement, practitioners may be hesitant 
to change their current practices, skeptical about the measures used, or unsure where to start. The 
information in this guide addresses these concerns, explores examples and lessons learned from 
the first phase of the project, and provides recommendations for how to approach each step of the 
process. Beginning with the rationale behind changing assessment practices, the guidance outlines 
what an MMA system is, what it takes to get started, the personnel involved, and the costs that may 
be incurred in the design and planning processes. States and college administrators alike will find 
tools for understanding the scope of work involved in setting up a new system that addresses the 
placement needs of their students and that aims to provide students with a better chance to succeed. 
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PREFACE 


tudents seeking the academic and technical skills required in the current labor market of- 

ten rely on community colleges. But these institutions bear the additional responsibility of 

preparing many of their students — some of whom have been out of school for years — for 
college-level course work, typically by placing them in developmental courses that carry no college 
credit. More than half the community college students who place into developmental education do 
not make it to graduation. Yet recent research suggests that many of these students may, in fact, be 
able to complete courses at the college level. Educators are asking whether the traditional reliance 
on a single placement test is poorly serving these students. 


To evaluate the predictive validity of single placement tests compared with “multiple measures” — 
the use of high school grade point average, psychological assessments, or other measures — MDRC 
has teamed up with the Community College Research Center (CCRC), which carried out the foun- 
dational research in this area. MDRC and CCRC visited the Great Lakes region from 2015 to 2016 
to better understand colleges’ interest in using multiple measures for placement; subsequently, the 
study team launched the first phase of the Multiple Measures Assessment Project in ten colleges in 
Minnesota and Wisconsin in fall 2016. 


For institutions interested in developing, implementing, or testing a multiple measures system, this 
guide presents critical information, questions, and lessons gleaned from those efforts: on gauging 
institutional readiness, on the importance of involving the faculty in placement criteria decisions, 
on integrating the new measures into school systems, and on the adviser-student conversation about 
placement results, among other considerations. 


The next phase of the project consists of a large randomized controlled trial of multiple measures 
assessment in five of the pilot colleges. In addition, MDRC and CCRC researchers, under the feder- 
ally funded Center for the Analysis of Postsecondary Readiness, are currently evaluating multiple 
measures for placement at seven colleges in the State University of New York system. In combina- 
tion, the findings from these projects will contribute causal evidence of the effects of using multiple 
measures placement on students’ completion of college courses. 


It is important to ensure that prospective students are afforded the best opportunity to succeed in 
their course work and earn a degree. Improving placement testing by integrating a multiple measures 
approach seeks to place students at a level at which they can succeed without diverting them into 
unnecessary courses that delay or even derail their progress. 


Gordon L. Berlin 
President, MDRC 
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INTRODUCTION 


Most community colleges admit all students who apply for admission — however, the vast majority 
then require students to demonstrate specified levels of literacy and numeracy before they can take 
college-level courses.’ Typically, students have been assessed using a single placement test, such as 
the College Board’s ACCUPLACER. Colleges, or sometimes states, choose “cut scores” on the assess- 
ment. Students who score below the cut score are not considered to be “college ready” in the relevant 
subject area and are referred to remediation, also known as developmental education. Research has 
shown that this method misplaces many students into developmental courses when they could be 
successful in a college-level course in the subject (underplacing) or places students in college-level 
courses who would have had a better chance of success if they had taken developmental courses 
(overplacing).? Thus some students must spend time and money on courses that they do not need, 
and others fail a course for which they were not prepared. As a result, colleges are seeking more 
reliable ways of assessing student readiness for college courses. 


One approach is to use additional indicators of student preparation, such as high school records, 
noncognitive assessments that examine behaviors and attitudes, specific assessments of writing or 
computer skills, or other test results. Combinations of such options are referred to as multiple mea- 
sures assessment (MMA) placement strategies. This guide will discuss how to begin the process of 
creating them within college systems. The information presented here is primarily informed by the 
Great Lakes Multiple Measures Assessment Project, a research project conducted from 2016 to 2018 
by two organizations — MDRC and the Community College Research Center (CCRC) of Teachers 
College, Columbia University — with funding from Great Lakes Higher Education Corporation. 
Technical assistance and implementation work done by the same organizations under the Institute 
of Education Sciences-funded Center for Analysis of Postsecondary Readiness (CAPR) also informed 
this work. (Box 1 provides a summary of the project and a related project under way from CAPR.) 


About This Guide 


In an effort to mitigate underplacement, the Great Lakes Multiple Measures Assessment Project 
helped ten colleges in Minnesota and Wisconsin to build multiple measures placement systems.? 
Students identified as needing developmental education based on test scores could be “bumped up” 
to a college-level course based on information such as high school grade point average (GPA) or 


1. Bailey, Jaggars, and Jenkins (2015). 


2. In one community college system study, Scott-Clayton, Crosta, and Belfield (2014) found that many students may be 
misplaced. Using the richest set of predictors available, the study calculated a “severe error rate” of students being 
misplaced, ranging from 24 percent in math to 33 percent in English. In particular, underplacement was a problem, 
accounting for the majority of the error rate. That study also found that high school grade point average (GPA) was 
more predictive of outcomes for the first college-level English and math courses than were placement test scores. 

A combination of placement test scores and high school achievement measures yielded the greatest explanation of 
variance. Belfield and Crosta (2012) had similar findings at another community college system. 


3. Five colleges were in Minnesota; the other five were part of the Wisconsin Technical College System. A list of 
participating colleges and their enrollment, retention, and graduation rates can be found in Appendix A. 
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BOX 1 


The Great Lakes Multiple Measures Assessment Project 
and the CAPR Assessment Project 


Researchers from MDRC and the Community College Research Center (CCRC) of Teachers 
College, Columbia University, are undertaking two random assignment studies to provide 
insight into whether changes in placement have an impact on student outcomes. 


This brief discusses lessons learned from the Great Lakes Multiple Measures Assessment 
Project, which MDRC and CCRC are conducting with funding from Great Lakes Higher 
Education Corporation. Phase | of the project, which began in 2016, consisted of a pilot at ten 
colleges in Minnesota and Wisconsin. In Phase Il, five of the pilot colleges are participating 

in a large-scale randomized controlled trial to learn the impacts of MMA on student success 

in college-level courses. In 2019, MDRC and CCRC will share findings on the colleges’ 
experiences, as well as early impact findings on course placements and pass rates for the first 
cohort of students, who entered the study during their first semester at the Phase II colleges. 
Final impact findings will be published in 2021. 


In the second project, MDRC and CCRC researchers with the Center for Postsecondary 
Readiness (CAPR) are conducting a randomized controlled trial of an algorithm placement 
system at seven colleges of the State University of New York (SUNY). An upcoming report will 
present early findings on the first-semester impacts on placement and completion of college- 
level courses among the first cohort of students randomly assigned to MMA algorithms at five of 
the colleges. Final impacts in the CAPR assessment study will be available in 2020. 


the results of a noncognitive assessment. To support this work, MDRC and CCRC offered technical 
assistance and conducted implementation research.* 


This guide, based on lessons learned from a pilot of these new systems, is designed for states and 
postsecondary institutions that are considering, developing, or implementing an MMA placement 
system. Following a discussion of the evidence for MMA and an outline of specific options, guidance 
is organized along the following lines: 


¢ How do we decide whether our institution is ready for MMA? 
¢ How should we organize the work? 


¢ How do we choose placement criteria? 


4. Specific technical assistance activities for the Minnesota and Wisconsin colleges included an initial convening of the 
colleges to begin planning for the development of an MMA system; assistance with selecting the most appropriate 
measures and placement strategy at each of the colleges; advice on implementation of the new MMA systems at 
each of the colleges; and a review of piloting procedures and results using the new MMA system for a small subset 
of traditional and nontraditional students. 
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¢ How do we design the logistical aspects of the MMA system? 
¢ How can we make sure that the system works as intended? 


e What are the initial costs of MMA design and planning? 


Why Change Assessment Practices? 


Large numbers of students enter college only to learn that they must begin their postsecondary ex- 
perience in developmental (remedial) courses for which they earn no college credit. Some students 
are placed into a sequence of developmental courses in each subject. Nationally, about 60 percent of 
incoming freshman are judged, by prevailing assessment systems, to require developmental instruc- 
tion in English, reading, math, or all three; the vast majority of these students are concentrated at 
nonselective two-year or less selective four-year colleges.® 


Educators need accurate information about students’ strengths and weaknesses to best position 
them for success. Developmental courses may be effective for some students, but they come at a cost. 
While financial aid such as federal grants or loans can be used to pay for developmental course work, 
developmental credits do not count toward degree requirements. Students who can be successful 
in college-level courses should not be required to spend time and money in courses that they do 
not need. In addition to the problem of unnecessary expense, being labeled “not college ready” may 
further discourage students who already doubt their academic ability. This may explain why many 
students fail to show up for their assigned developmental courses after being assessed.® 


Does Multiple Measures Assessment Work? 


Emerging research suggests that the use of MMA can have a positive effect on student outcomes. 
While no rigorous experimental studies are complete, evidence from a pilot and early results from 
a randomized controlled trial are promising. 


Long Beach City College (LBCC) in California piloted an assessment method based on information 
from high school English and math grades, GPA, and state exit exam scores. LBCC’s internal evalu- 
ation found that students in the pilot were more likely to complete transfer-level math and English 
in the first term than fellow LBCC students who did not participate in the pilot.’ Other California 
colleges are currently in the process of replicating aspects of the Long Beach model. While these 
results are promising, the effects were estimated from an analysis in which students who self-selected 
into the program are matched with similar students who attended LBCC before the pilot, leaving 
some possibility of selection bias affecting the magnitude of the effects. 


A random assignment study is currently being conducted by CCRC and MDRC under CAPR at seven 
State University of New York colleges (Box 1). The researchers worked with the colleges to develop 


5. National Center for Public Policy and Higher Education and Southern Regional Education Board (2010). 
6. Steele and Aaronson (1995). 
7. Long Beach City College, Office of Institutional Effectiveness (2013). 
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algorithms based on historical data to predict students’ probability of success in college-level courses. 
Early findings show that students who were placed using these algorithms were more likely to place 
into college-level English and math and more likely to enroll in — and pass — college-level English 
and math courses in their first semester, compared with the business-as-usual group.® While longer 
follow-up is needed to understand how students placed with traditional methods will fare after tak- 
ing the developmental education sequence, these positive impacts are a promising start for students 
placed with multiple measures. 


Five of the ten colleges participating in the pilot of the Great Lakes Multiple Measures Assessment 
Project are moving forward with a large-scale randomized controlled trial that began in spring 2018. 
This second phase of research will provide additional evidence on the effects of MMA placement 
systems on student outcomes. 


MMA Placement Strategies 


Key decisions must be made during the design of MMA systems on (1) what measures to use and (2) 
how to combine the various measures to arrive at a placement decision for each student. 


Commonly Used Alternative Measures 

Colleges have many choices of measures and typically select them based on their predictive valid- 
ity, availability, ease of use, and cost, factors discussed in greater detail below. The primary options 
used by colleges — traditional placement tests, other assessments and questionnaires, high school 
transcript information, and standardized test results — are listed in Box 2.° There is strong empiri- 
cal evidence that high school GPA is one of the best available predictors of college success.'° One 
advantage of high school GPA is that it is an aggregate measure of performance over multiple years, 
unlike a one-time assessment. It reflects not only content knowledge, but also behaviors, such as 
attendance and participation, that influence success in college. Previous course work from high 
school or college transcripts may also have some value in placement decisions. While course work 
has not been found to be as predictive as high school GPA, it can be used in combination with GPA 
to address concerns about subject-specific knowledge. 


Noncognitive assessments, such as the Learning and Study Strategies Inventory (LASSI), ACT 
Engage, and the Grit Scale (Box 2), have the potential to take into account some of the other factors 
that influence college success, such as motivation and problem-solving skills. While the evidence is 
less strong than the evidence for high school GPA, there is research showing that some noncognitive 
assessments are predictive of success in college courses." 


Placement System Options 
Once the measures have been selected, the resulting information must be integrated in some way to 
yield a placement for each student in each subject area. Figure 1 shows three examples of placement 


8. Barnett, Bergman, Reddy, and Roy (forthcoming). 
9. Barnett and Reddy (2017). 
10. Belfield and Crosta (2012); Scott-Clayton, Crosta, and Belfield (2014). 


11. Appendix C provides a list of references on noncognitive assessments. 
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BOX 2 


Primary Options for Assessment Measures 


MEASURES ADMINISTERED BY THE COLLEGE 


1. Traditional placement tests (Such as ACCUPLACER) measure students’ math, reading, and 
writing skills. 


. Noncognitive assessments measure attitudes and behaviors that have been found to 
be relevant to college success. Commonly used assessments are the Learning and Study 
Strategies Inventory (LASSI), which has ten scales to measure components of a student’s 
approach to learning; ACT Engage, developed by the ACT assessment organization to 
measure such qualities as motivation, social engagement, and self-regulation; and the Grit 
Scale, which focuses on perseverance and consistency of interests. 


. Writing assessments 
. Computer skills assessments 


. Questionnaires (may include self-reported high school transcripts) 


MEASURES OBTAINED FROM OUTSIDE THE COLLEGE 
1. High school grade point average (GPA) 


2. Other high school transcript information (courses taken, course grades, class rank, years 
since graduation) 


. Standardized test results (for example, ACT, SAT, Smarter Balanced, or PARCC 
assessments). In addition to the ACT and SAT college admissions tests, colleges may use 
scores from other tests administered by high schools: for example, systems such as Smarter 
Balanced and PARCC (Partnership for Assessment of Readiness for College and Careers), 
which some states have adopted to meet federal accountability requirements. 


systems. The top diagram represents what happens at most colleges that do not use MMA: Some 
students are exempt from the placement process on the basis of standardized test scores or previous 
course work. Students without exemptions take a traditional placement test such as ACCUPLACER. 
The middle diagram shows an example of a decision rule system, which takes into account multiple 
measures (high school GPA and a noncognitive assessment) as well as the placement test; a system 
may use different measures from those shown in the example or use those measures in a different 
order. Finally, the bottom diagram shows an example of a decision band system. Students with 
placement test scores above or below the band are placed as they would have been in a traditional 
placement system. Those whose tests fall within the band have their other measures considered. As 
with the decision rule example, the measures used and their order may vary. 
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While not shown in Figure 1, an algorithm approach would look similar in a simplified diagram to a 
typical placement process, but the multiple measures algorithm, rather than the ACCUPLACER score, 
would generate the placement decision. Such models may be somewhat more precise than the decision 
rules shown here, but considerable historical data preparation and analysis is necessary to develop 
an algorithm, and implementation is considerably more complicated than it is for decision rules. 


Whatever method is used, many colleges treat the result as guidance rather than a final placement 


and allow students to make their own decisions about what courses to enroll in based on the informa- 
tion (known as directed student self-placement). Box 3 summarizes the placement system options. 


BOX 3 


Placement System Elements and Options 


EXEMPTIONS OR WAIVERS: Students are placed directly into college-level courses without the 
need for placement testing if their scores on specified tests or other measures exceed a certain 
threshold. 


DECISION RULES: As shown in the middle panel of Figure 1, a sequence of rules compares 
each selected measure to a threshold in a predetermined order. If the threshold is met, a 
placement is generated; if not, another rule is applied. 


DECISION BANDS: As shown in the bottom panel of Figure 1, decision rules apply only to 
students who fall within a certain range on a specified indicator (Such as high school grade point 
average or a placement test score), usually just below the cutoff. 


PLACEMENT FORMULA (ALGORITHM): An algorithm applies a weight for each of various 
factors based on an analysis of historical data to calculate the probability of success in college 
courses and generate a recommended placement. 


DIRECTED SELF-PLACEMENT can be used in conjunction with any of the above methods, 

or on its own. When this is used with another method, the student is told of the generated 
placement but given the option to enroll in either developmental or college-level courses. In a 
system where no definitive placement is given, the student has a conversation with the adviser 
or counselor about test results, prior courses, and grades, and selects preferred courses. 


Introduction | 7 


CONSIDERING MMA, STEP BY STEP 


How Do We Decide Whether Our Institution 
Is Ready for MMA? 


Whether a college should adopt MMA depends on state or system policies; the views of the faculty, 
the staff, and college leaders; and logistical factors. For example, do current mandates conflict with 
MMA, or leave little to be gained? Do faculty members recognize a need for improvement in the 
placement process? Can new measures be easily obtained and integrated into the school’s data system? 


4 Background 
The research conducted in Minnesota and Wisconsin indicated that colleges vary in their readiness 
to adopt a reform such as an MMA placement strategy. Some differences involved institutional readi- 
ness to adopt reforms in general; others involved conditions favorable or less favorable to initiating 
MMA specifically. 


For the most part, however, conditions were supportive for MMA implementation, and many col- 
leges were engaged in discussions about how MMA would fit with other efforts to improve student 
outcomes. As one Minnesota college administrator stated, “In multiple measures, it’s part of the 
larger context of how do we think in a more nuanced way about bringing students into the college 
and making sure they’re set up to be successful in the programs of study.” 


Across the board, college leaders were committed to doing this work. In some cases, they had con- 
cluded that it was the right thing to do; in others, they were influenced by national trends toward 
increased use of MMA. Leadership commitment was extensive in Minnesota, where the state required 
MMA placement systems to be set up within a few years. 


Many of the colleges had multiple reforms under way; for most of them, simultaneous changes ap- 
peared to be a normal way of doing business. Some colleges had existing structures in place (task forces 
or committees), which facilitated the process. In these settings, it was easier to make decisions and 
move toward development of an MMA system than it was where those structures were not in place. 


The biggest obstacle to MMA implementation noted by interviewees was access to students’ high school 
transcripts. While some colleges asked students to submit transcripts at the time of application, this 
practice was not universal, and some colleges that did require transcripts had no automated process 
to pull the information into their data systems. Some colleges were reluctant to require students 
to submit transcripts, considering it to be a barrier to entry. This may be a concern particularly in 
the case of adult students applying to college many years after graduating from high school. Other 
obstacles to implementation of MMA included faculty concerns about the prospect of providing too 
little developmental education to those who need it, or about the alternative measures themselves; 
uncertainty about technological readiness; and college systems that were already stretched thin due 
to staff changes or other commitments. 
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Recommendations 

Administrators should review pertinent state and college policy before beginning the MMA develop- 
ment process. Legislation may require the use of specific placement measures, establish exemptions 
from developmental education, and set reform timelines.'* Developmental education reforms under 
way at the state or college level, such as the use of corequisite courses, may need to be factored into 
MMA placement rules. Corequisite models of remediation may already place large numbers of stu- 
dents into college-level courses; if so, MMA would not necessarily increase the number of students 
in college-level courses so much as it would reduce the number in developmental corequisites. 


After taking stock of the current state of developmental education initiatives, colleges can gauge 
institutional readiness. Because many people, both faculty and staff members, are involved in building 
a successful MMA system, project leaders should assess staff views about current placement methods, 
their commitment to changing them, and the college’s technical capacity to do so. Project leaders 
may want to consult the criteria below, which are specific to the use of a new assessment system, 
and which emerged from MDRC and CCRC work on this project and other MMA research.'* These 
conditions are identified as contributing to a smoother implementation process: 


e There is a consensus that placement tests alone are not good predictors of student success in college. 


¢ The faculty is committed to making sure that the right students are placed into developmental 
education. 


e The college leadership supports the use of a more effective assessment process. 
¢ The college has reviewed and discussed the research on the use of MMA. 
¢ The college has access to incoming students’ high school records. 


e The institution has a strong information technology (IT) capability. 


How Should We Organize the Work? 


Once the decision has been made to embark on developing a new MMA system, leaders must assign 
responsibility for different aspects of the work. While drawing on the institutional practices that have 
worked in the past, colleges should weigh various factors: How much autonomy does the institution 
have? Is a large committee needed to balance the workload, or would a small one be more efficient? 
What is the best way to keep campus divisions informed and involved? 


Background 

College systems vary by state. Minnesota colleges in the pilot were following a state-mandated de- 
velopmental education reform plan, whereas the Wisconsin Technical College System is less central- 
ized. Interviews and observations conducted at Minnesota colleges found that, while there was an 
organizational structure at each college responsible for working on the MMA initiative, there was 


12. Appendix A provides a summary of the state policy context for the colleges in the pilot. 


13. For example, Barnett, Bergman, Reddy, and Roy (forthcoming). 
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also considerable communication between colleges and the college system office as they collaborated 
to create a statewide MMA system. In Wisconsin, most of the work related to the development of 
MMA systems was done at the college level, although there were some opportunities for mutual 
learning at statewide and regional events. 


All the colleges involved in the project set up committees to advance the work. In some cases, the 
committees were fairly large, inclusive, and organized. For instance, one college formed a core team 
of 12 to 15 staff members, ensuring that all areas of the college affected by the new placement system 
were represented. The committee had a “sponsor,” the college president, and a “charge,” which delin- 
eated their responsibilities to the college. In addition to their deep engagement in project planning 
and oversight, representatives would make sure that their respective departments were informed of 
project updates via in-person meetings and frequent email communication. In other colleges, the 
core committee working on the project consisted of one or two people, who did most of the work but 
brought in others as necessary to weigh in on certain decisions or perform specific tasks. In these 
settings, the belief was that it was easier to have a small, nimble group rather than a large group with 
complex schedules and other responsibilities. 


In most cases, project leaders had spent time talking about the new assessment system and the pilot 
with people throughout the college community. Some also had discussions with people in the K-12 
educational system, especially in cases where there were large high school-college dual enrollment 
programs that used placement testing. Many colleges scheduled meetings with a range of stakeholders 
before the pilot, often to address concerns associated with a change in assessment criteria. As one 
project leader stated, “There were many, many steps because we had to get a lot of people to buy into this. 
... it’s really the program’s deans and associate deans . . . that were probably, initially, the most concerned 


about this project.” 


»> Recommendations 
The assignment of responsibilities generally involves establishing a committee, clarifying which 
college leaders will provide oversight, and making sure that lines of communication are open with 
those who will be affected by the change — which includes almost all divisions on campus. It is 
recommended that colleges use approaches that have worked for them in past college-wide reforms. 


Faculty participation and early endorsement of the process is essential for MMA to be implemented 
in a sustainable way. All departments touched by MMA should be involved in discussions and plan- 
ning from the start, but it is essential that faculty be included at the earliest stages. Because faculty 
members are most familiar with their course content and have insights into which students are likely 
to pass, they may not be comfortable with placement criteria that they did not contribute to. Faculty 
members must provide input on what measures are to be used and how the decision rules are to be 
defined. It is also important to educate the faculty about what MMA is and what advantages and 
potential challenges are to be expected. These may not always match preconceived ideas about MMA. 


Colleges may underestimate the number of staff members affected by the new MMA system. As dis- 
cussed in the section on logistics, several departments — recruiting, admissions, advising/counseling, 
testing, information technology (IT), institutional research, the registrar, and the faculty — need 
opportunities to learn about the proposed system and provide their point of view. 
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How Should We Choose the Placement Criteria? 


Historical data, if available, can guide decisions about placement criteria; otherwise, the literature 
on high school GPA and noncognitive assessments provides a basis for decisions — which should be 
informed by recommendations from faculty and staff members. Which measures are most predic- 
tive, and which can be obtained? How should the sequence of assessments be structured? Which cut 
scores are most likely to promote student success? 


Background 

Colleges in Minnesota and Wisconsin all incorporated high school GPAs and noncognitive assess- 
ments, along with ACCUPLACER results, into their MMA systems. Other measures, such as specific 
courses taken in high school, were considered but not added during the pilot project. In addition, 
they all used decision rule or decision band approaches to integrate the different measures. Colleges 
agreed that additional measures could only be used to “bump up” students: In a system where stu- 
dents took the ACCUPLACER test first, measures such as GPA or a noncognitive assessment could be 
used to raise them a level, but not to place them lower. 


Among the challenges in choosing placement criteria were internal disagreements on which measures 
to use. Some colleges and individuals had little faith in high school GPA, especially for older students, 
while other colleges had more confidence in its ability to predict college achievement. High school 
GPA seemed to be of particular concern among faculty members because of perceived grade infla- 
tion in local high schools. Other faculty members questioned the rigor of the high school courses 
students typically took, particularly in math. In many cases, people became more comfortable as 
they learned more about the considerable amount of research that has been done on the value of the 
high school GPA in particular. 


Similarly, opinions differed on the use of noncognitive assessments. Some staff members thought they 
would allow for the inclusion of additional, important dimensions in the placement decisions, while 
others were somewhat skeptical. Furthermore, some colleges questioned the accuracy of noncogni- 
tive tests if students completed them outside the testing center or if students intentionally selected 
answers they thought would provide desirable results. One college representative noted, “We don’t 
want people to go into the ACT Engage thinking, I should answer this way because I want to move 
up. We want them to be honest taking it.” 


During the planning process, colleges created flow charts to help them visualize their decision 
rules and the order in which the rules would be used. Colleges shared their flow charts with each 
other and discussed ways to address state-specific issues, then made further changes individually 
to reflect local conditions and preferences. College representatives were emphatic that peer learning 
had helped them make good choices. 


The placement approaches taken by each of the participating colleges are presented in Appendix 
B. Key differences included cutoffs for high school GPA and test scores in each subject, the choice 
of noncognitive assessment, and the order in which different measures were considered. It should 


14. For example, Hetts (2016). 
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be noted that these were preliminary decisions made for use in the pilot phase of this project, and 
some were later adjusted. 


» Recommendations 
College institutional research departments should begin by looking for any historical student data 
on the measures being considered. Did students with a high school GPA above a certain threshold 
perform better than others? A good starting place for these analyses of high school GPA is the grade 
deemed acceptable in college-level courses. For example, the data may indicate that nearly all stu- 
dents who earned a C or better in a college-level course received at least a C+ average in high school. 


In the absence of school data, prior research can be a guide. Decision rules used in California 
and North Carolina have consistently set high school GPA cutoffs in the 2.5 to 3.0 range, based on 
extensive research.'® (In those states, when students enroll in college directly from high school, 
eleventh-grade GPA is used if twelfth-grade GPA is not yet available.) While the most accurate 
source is the high school transcript, there is evidence that students self-reported high school GPA 
reliably in California data."® 


For measures with less data or research backing them up, the college may want to begin collect- 
ing data on a trial basis to learn more about how they correlate with student outcomes. Both the 
predictive value of the measure and the feasibility of obtaining the data will need to be taken into 
account, as well as logistical considerations such as cost and time required for testing. Selected fac- 
ulty and staff members should have a chance to weigh in. 


When choosing a noncognitive assessment, there are several questions of fit and logistics to 
consider. Box 4 enumerates some criteria used by colleges in the pilot project, with guidance from 
CCRC and MDRC. It may make sense to use the results of subscales related to particular character- 
istics rather than the full instrument. For example, on the Learning and Study Strategies Inventory 
(LASSI), an instrument used by several of the colleges in Minnesota, scores on the motivation scale 
were correlated with student success, so that subscale was selected for inclusion in the MMA place- 
ment strategy at a number of colleges. In the absence of historical data, information may be available 
from assessment developers about which scales indicate college readiness. Students who fall just 
short of college readiness by traditional tests — or who are missing other measures — who score 
high on these scales may be good candidates for bumping up to college level. Once assessments are 
administered, the institutional research department can begin to accumulate data on how student 
outcomes correlate with the subscales. 


Once measures are selected, colleges will need to determine their order in the placement structure 
and the cut scores for each measure. The following steps may help determine how to combine the 
measures: 


¢ Create a diagram of the college’s current system of placement for each subject, including all ex- 
emptions or waivers, as in the top panel of Figure 1. 


15. MMAP Research Team (2017); North Carolina State Board of Community Colleges (2014). 
16. Sanchez and Buddin (2016). 
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BOX 4 


Noncognitive Assessment Considerations 


Several criteria should be considered when selecting a noncognitive assessment: 


1. FACE VALIDITY (fit with college priorities, faculty judgment on the importance of the 
measured domains, expected usefulness of results) 


2. PREDICTIVE VALIDITY (in relation to college success and completion of college-level 
English and math courses) 


3. COST (both initial and ongoing) 


4. TIME REQUIRED (for both staff members and students to spend on the assessment and 
follow-up activities) 


5, FIT WITH COLLEGE SYSTEMS (existing policies and procedures related to admissions, 
testing, counseling, registration, and IT) 


6. SPECIAL CONSIDERATIONS (population served, goals of the placement process, college 
context and history, existing course sequences) 


¢ Create a diagram incorporating the new measures, showing the order in which the measures will 
be considered and the decision rules that will apply (see Figure 1 for options). 


¢ Take time to consider, with faculty members, the cut scores that are most likely to increase stu- 
dent success. Think further about how the cutoffs used may change the number of students in 
each course. 


e Repeat the process as necessary to cover all programs at the college and their prerequisites. 


e Review the new rules with key players on campus and make adjustments as needed. 


How Do We Manage the Logistical Aspects of the 
MMA System? 


Adoption of an MMA placement system means acquiring data, administering tests, and integrating 
the results into the current data system, as well as considerations of how the changes will affect stu- 
dents, faculty members, and advisers. Recommendations address the roles of admissions, registrar, 
testing, and IT staff, and advisers, faculty, and administrators. 


Background 

The ten colleges in Minnesota and Wisconsin piloted their approaches to MMA in the spring and 
summer of 2017. The goal of the pilot was to test each level of the new placement process, consider 
ways to automate the system, and determine the extent to which the approach would actually change 
student placements. Here are a few of the lessons learned: 
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Colleges need a process not only for obtaining high school GPA but also for integrating it into the 
system. Three of the colleges had routinely obtained high school transcripts for students admitted to 
the college. At another college, large numbers of students had taken dual enrollment courses while in 
high school, which gave the college access to their high school records. Other colleges did not have 
a process in place for obtaining transcripts before the pilot. Even when transcripts were received, 
however, the data were not routinely entered into the college’s data system in a form that would allow 
them to be used for placement. As a result, a number of colleges were considering whether to allow 
students to self-report their high school GPAs rather than requiring transcripts upon entry. Those 
not requiring submission of a transcript were often concerned about adding barriers to admission. 


Coordinating the testing can be tricky. Most students placed using MMA were required to take either 
the placement test (ACCUPLACER) alone or both the ACCUPLACER test and a noncognitive assess- 
ment. Among the colleges that elected to use the Grit Scale as their noncognitive assessment, the 
process was relatively easy, as the 8 to 12 items in the scale could be included within ACCUPLACER 
itself. For the colleges that used ACT Engage or LASSI, the testing center had to figure out a way to 
sequence the two tests and assist students as they finished one and began the other. Alternatively, 
students were sent a link to one of these assessments and asked to take it before coming in for test- 
ing; however, many students did not do so and had to be given the test upon arrival at the college. 
Neither system was ideal, and during and after the pilot, college staff members continued to discuss 
better ways to administer these assessments. 


Advisers must be prepared to communicate placement results to students. Staff members at a 
number of the colleges were concerned that the student advising process would become more com- 
plicated with the new assessment and placement system, for several reasons. Rather than reporting 
individual test scores, advisers would need to explain the multilayered rationale for the placement 
students received. In addition, several colleges allowed considerable latitude to advisers and/or stu- 
dents to determine which placement option to accept. Colleges had to decide whether that latitude 
would continue in the new system. 


The process involves demands on a variety of staff members. Through the piloting process it was 
clear that each affected department or group within the colleges faced its own challenges in setting 
up the MMA placement strategy. Box 5 presents some important issues likely to arise for offices of 
admissions, testing, IT, institutional research, and the registrar, as well as the faculty. 


Recommendations 
Once the system is designed and mapped, there is still considerable work left to do to develop a 
logistical flow that will meet the needs of students as well as those of the college staff and systems. 


The admissions staff, the testing staff, and the registrar should plan to address the following tasks: 


e Determine how to obtain high school information. Options include getting transcripts from 
students or their high schools or allowing students to self-report. Efforts to obtain transcripts 
smoothly and quickly can benefit from good communication with counseling offices at the main 
feeder high schools. In some cases, high schools automatically send the transcripts of all students 
who express interest in attending the college. Determine how to upload them into the testing 


14 | A Better Shot at Success 


BOX 5 


Potential Challenges of MMA for Divisions 
Involved with New Student Enrollment 


Admissions or Registrar 

e Adjusting to increased data entry demands 

e Obtaining high school transcripts before students enroll 

e Timing the entry of high school data into the system before placement 


e Communicating new requirements without discouraging student enrollment in the college 


Testing 

e Building a new set of rules to incorporate multiple measures, in concert with IT 

e Providing students with information about the revised assessment procedures and results 
e Managing the administration of both ACCUPLACER and a noncognitive assessment 


e Importing new information into the college’s data system 


Information Technology (IT) or Registrar 
e Creating new fields in the management information system 


e Incorporating different kinds of measures into a data system and applying placement rules 
without requiring additional staff time 


e Adapting registration blocks or prerequisites to reference the revised placement results 
instead of raw test scores 


Institutional Research 


e Providing analyses to inform the development and test the efficacy of MMA systems 


Advising/Counseling 


e Understanding and becoming comfortable with the new assessment system, compared with 
test scores that appear to have a more definitive result 


e Preparing ways to communicate results to students; depending on the method used, 
placement results may not be as transparent as test scores 


Faculty 
e Resolving concerns about changing standards for college-level enrollment 


e Rethinking the meaning of college-ready: not only those most likely to succeed, but 
also those more likely to succeed by enrolling in college-level courses than by taking 
developmental courses first 
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system or management information system (MIS). Will they be entered manually, or be uploaded 
as a file? Who is responsible for each step? 


* Communicate with incoming students and high school counselors about any new requirements 
or policies, such as the use of exemptions or waivers, testing requirements, the need to prepare 
for tests, and so on. 


¢ Ifa noncognitive assessment is to be administered, plan the timing of its administration and the 
entry of its results into the placement system. 


¢ Make sure that the MMA placement is recognized by the system’s registration blocks or prereq- 
uisite requirements so that students who are bumped up can register for the appropriate courses. 


¢ Develop a placement report for each student that is easy to interpret. The goal should be to convey 
the placement result, not raw scores on tests that will no longer be the sole determinant of place- 
ment. If students and advisers simply receive all the individual measures and try to make sense 
of them, the rules may not be applied correctly and placement results are likely to be inconsistent 
from student to student. Even in a directed self-placement environment, a clear placement result 
based on institutionally approved rules will help students make their decisions. 


The IT department should be included in conversations about these options to improve efficiency 
and the reliability of the measures captured. The IT staff should advise on how to automate as 
much of the new placement system as possible: how to integrate it within the current data system 
and how to ensure that the necessary data are extracted, stored, and available for use with as little 
human intervention as possible. Because IT departments often have workloads determined months 
in advance, notifying IT early about plans to make the transition to MMA is important to ensure 
that the staff will have the time to make adjustments and resolve any issues before the expected 
rollout of the new system. 


Some level of automation is required to maintain consistent standards for all students going through 
the placement process and to be able to place large numbers of students effectively. Multiple measures 
rules can sometimes be built into existing placement test systems, such as ACCUPLACER, yielding a 
placement result in a manner similar to that of the traditional testing process. However, student data 
on measures like high school GPA and noncognitive assessments must be made available to these 
placement systems, usually through an upload process, which may be cumbersome. 


In some cases, it’s preferable to work with the college IT staff to build the MMA rules into the school’s 
MIS, where the measures are likely to be entered and stored. Automation of placement is already the 
norm at most colleges that have a traditional placement test in place. When the test score is recorded, 
it is usually automatically fed into the MIS in a way that blocks registration for any courses above 
the student’s placement level in math or English. This is also what MMA can look like: Students’ 
scores on the measures are entered or uploaded into the MIS, and a set of logical rules populate a 
placement field with the MMA placement. In the case of decision rules, this is usually an if-then 
statement referencing other fields. In the case of an algorithm, other software may be required to 
calculate the probability of success or the appropriate placement. 
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Advisers and counselors should create clear guidelines for communicating with students about 
their placement results. As a departure from the long-established placement test system, MMA re- 
quires a cultural change around the adviser-student placement conversation. Because of the clarity 
and familiarity of placement test scores used in the past, advisers may struggle at first to treat MMA 
placement as the “correct” placement. Before the new system is operational, advisers should receive 
information about the predictive power of measures such as high school GPA, and they should be 
involved in crafting the language used to communicate with students about the basis for placement 
results under MMA. 


Once the placement can be reliably generated in a timely manner for all incoming students, the 
messaging should be consistent. Advisers must have clear MMA placement results, and not just the 
results of the measures themselves. While students should be given some basic information about 
what informed their placement, they do not need to be told how their MMA placement compares 
with the previous test-only placement. Instead, advisers should communicate to students that the 
goal is to place them appropriately. 


Consider the implications for teaching responsibilities and student eligibility for services. Use of 
MMA may change course enrollment, including a possible reduced need for developmental courses. 
This may affect classroom size, demand for faculty qualifications for different kinds of courses, and so 
on. Students’ access to some services, such as federal TRIO programs (outreach and student services 
programs designed to identify and provide services for individuals from disadvantaged backgrounds), 
supplemental instruction, or other special programs, may be tied to their developmental education 
needs. Staff professional development offered at the beginning of the project can help ensure that 
staff members are comfortable with the roles they will play. 


Administrators may need to motivate staff members to put in the thought and effort that MMA de- 
sign requires. Administrators who hold staff members accountable to timelines and milestones are 
more likely to see MMA implemented as intended and on time. Administrators overseeing MMA 
development should try to attend as many of the early planning meetings as feasible and stay engaged 
as the process moves forward. 


How Can We Make Sure That the System Works 
as Intended? 


Colleges will want to know how their MMA placements compare with those under the traditional 
system. Are students registering for the right courses? Is there an appropriate number of course 
sections? Are different groups of students being affected differently? 


Background 
In the logistical pilot conducted by the ten colleges in Minnesota and Wisconsin in the summer 
of 2017, MMA appeared to make a difference in how students would be placed.’ The numbers in 


17. The ten participating colleges were not required to change students’ actual placements during the pilot, only to 
generate their MMA placements in real time so that it would be logistically feasible to apply them if desired. While 
most colleges did not choose to change real student placements in this period, a few did so. In the second phase of 
this project, MMA placements will be applied, and student outcomes will be tracked. 
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FIGURE 2 


Percentage of Students Placed in 
College-Level Courses, by Placement System, 
2017 Pilot at Ten Colleges 


College-level 29% 


College-level 57% 


ee 


Test only HE Multiple measures 


SOURCES: Placement data collected from the ten Minnesota and Wisconsin colleges participating in the 
pilot. 


NOTES: Most colleges ran their pilots for a week or so in June or July, with the exception of one Wisconsin 
college that continued its pilot for several months, yielding much higher sample sizes. 
Colleges are equally weighted in the averages displayed. 


Figure 2 represent, on average, what would have happened if students had been placed under MMA 
compared with what would have happened if they were not. While there was variation in the numbers 
across colleges, there were some clear themes. When using ACCUPLACER alone, over half the pilot 
students placed into developmental courses in math, and nearly half did so in English. In both math 
and English, placements generated using the MMA system were much more likely to put students 
into college-level courses, as intended. On average, across all ten colleges, an additional 28 percent 
of students placed into college-level math, and an additional 17 percent placed into college-level 
English under the MMA pilot rules. 


»> Recommendation 
Once MMA placements are being administered, either in a pilot or on a large scale, it is important to 
monitor the process and its results, especially at first. Regular check-ins should occur with advisers 
to get feedback on how meetings with students are going. Students’ actual course registrations should 
be checked against the placements generated by the automated system. Finally, to ensure that place- 
ments are not deviating from expectations in the number of course sections needed, colleges should 
periodically check that placements into the relevant courses roughly correspond to available seats. 


Colleges piloting their programs should analyze data on the measures and student placements. It’s 


a good idea to periodically calculate students’ placements under old test-only rules as well as the 
new MMA rules to monitor the percentage of students whose placement is being changed by the 


18 | A Better Shot at Success 


new system. This information is important for understanding whether MMA is making a differ- 
ence, as well as for planning future course sections. Finally, this is a good opportunity to look for 
any changes in the way different groups of students are placed, being especially mindful of equity 
considerations. For example, how do MMA placements of students of color compare with those of 
white students? Are there gender differences in placement rates under MMA? Are the gaps between 
these groups’ rates of placement into college-level courses shrinking, staying the same, or growing 
with the introduction of MMA? 


What Are the Costs of MMA Design and Planning? 


The MMA system requires resources — personnel, equipment, and facilities — from across the col- 
lege campuses, both for its design and setup and for its ongoing operation. 


The initial investment has three components: data collection, data analysis, and preparation of 
the staff and systems. First, to inform creation of placement rules, colleges or consultants must 
collect historical data on students’ characteristics, including high school transcripts, if possible; 
their placements based on traditional placement tests; and their subsequent college outcomes. These 
data are already available in some colleges, but in others, more extensive data collection is needed, 
and in some colleges it may not be feasible at all. Second, staff members can analyze internal data 
or external data from placement rules in other college systems, to determine a new decision rule 
based on multiple measures of student readiness (or a new placement algorithm). Third, resources 
must be allocated to make changes to internal computer systems, to train staff members in the new 
placement system, and to purchase any new instruments used to assign placement. 


Ongoing operations require modest resources relative to the status quo. Staff members must con- 
tinue to collect information from entering students for the new placement rules. In addition, staff 
time may be needed to use the MMA rules to assign students to developmental education or directly 
to college-level courses. This second ongoing cost may be avoidable with sufficient automation. 


The following estimate represents the expected cost of introducing MMA to a two-year college in 
Minnesota or Wisconsin, relative to the cost of “business as usual” placement. It covers the resources 
required for MMA with respect to (1) administrative setup and data collection for the assignment 
rules, (2) the creation of the decision rules, and (3) the application of the new rules at the time of 
testing. At present, information is not available on costs for administering placement tests, which 
may be slightly higher for MMA depending on the complexity of the new decision rules. Changes 
in enrollment affecting developmental and college-level courses may also involve costs, which will 
vary by college; such costs are not estimated here. 


The costs were calculated for the ten pilot colleges using the ingredients method:"* Costs were derived 
from the inputs (resources in hours of time or units of materials) used at each college, multiplied 
by standardized prices per input. (The use of standardized prices allows these cost estimates to be 
compared with estimates from other studies.) All costs are expressed in present value 2018 dollars. 


18. Levin et al. (2017). 
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Information on ingredients was collected from survey or interview information provided by staff 
members who implemented the new testing protocols. Information on input prices and overhead 
costs is from secondary sources. 


Table 1 presents the average cost per college. Resources for personnel to implement the new place- 
ment system are divided into several groups. Within the college, the groups are IT/computing staff, 
responsible for creating new computing and data infrastructure (and some data management); 
members of the testing office staff and senior staff (including faculty and registrars), responsible 
for creating the new decision rules and implementing them during and after the students take the 
placement tests, as well as advising students about courses; and administrative staff members, sup- 
porting the implementation of the new placement system. Outside the college, the personnel costs 
cover time from the evaluation team that helped advise on the new rules. These external costs are 
allocated evenly across the sites per state and are priced using local pay rates. If this pilot were to 
be replicated in other Minnesota or Wisconsin colleges without MDRC or CCRC input, this tech- 
nical assistance time would likely be incurred by someone else. It is possible that staff members at 
the state system-level offices also committed time to MMA; these costs are unavailable and thus 
excluded from Table 1. In addition to personnel costs, the estimate includes fringe benefits and all 
other operating costs (overhead and facilities). 


Across the five colleges in Minnesota, the average cost to introduce the new system is $48,520, with 
the costs varying from $24,150 up to $75,710. Across the five colleges in Wisconsin, the average cost 
to introduce the new system is $63,730, with a range from $50,550 up to $89,580. These costs accrued 
over the course of approximately one year, from the beginning of the project in September 2016 
through the end of the pilot in summer 2017. Colleges getting started with MMA placement should 
allow at least this much time to develop and pilot rules before trying to scale up the system. 


Many of the cost components are one-time costs, for creation of IT data systems and time allocated 
for program officers, senior faculty and staff members, and evaluators to decide on the specifics of 
the MMA system and undergo training on its use. Thus, operating costs should be lower than these 
estimates. But the estimates here are likely to be conservative: Some persons involved in MMA may 
not have been accounted for, and a decision rule may be more complex to implement than those in 
the pilot and may need to be revised within a few years. 


The resources required for designing and operating MMA are modest but not trivial. A cost- 


effectiveness analysis will follow in the next phase of research, when information on the outcomes 
of MMA is available, using the cost estimates presented here in combination with operating costs. 
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CONCLUSION 


This guide presents some key considerations for colleges contemplating multiple measures assess- 
ment: what first steps to take, how to choose appropriate measures to use and how they can be used, 
what challenges to expect, how to manage the logistics, and how to make sure the new system works 
as intended, as well as an idea of the costs involved. 


Students who need developmental education to succeed in college-level courses should be placed 
into developmental courses. However, because developmental education requires student time and 
expense, it may discourage some potential college students. It is important to ensure that those who 
could succeed in college-level courses get the opportunity to take them upon entry into college. 
The use of an MMA placement strategy should increase the chances that students will be optimally 
placed, which should then increase their chances of future success. 
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APPENDIX 


A 


The Policy Context in Minnesota 
and in Wisconsin 


Minnesota 

In 2017, the Minnesota Legislature required the Minnesota State Board of Trustees to prepare a plan 
to reform developmental education offerings. The resulting plan includes the following developmental 
education reforms: 

e Accelerated courses (combining two or more developmental courses into one) 


¢ Expansion of professional development for faculty teaching developmental education 


¢ System-wide multiple measures placement guidelines to be implemented by the start of the 2020- 
2021 academic year 


Earlier state-level placement reforms enacted over the last three to five years include the following: 


e An ACCUPLACER exam with additional questions designed to provide a weighted score that could 
potentially boost the raw score of a student on the “bubble,” just below the college-level cut score 


¢ ACCUPLACER exam waivers for students whose ACT, SAT, or Minnesota Comprehensive Assessment 
(MCA) scores demonstrate college readiness 


The weighted ACCUPLACER scores and the waivers continued to be used in the piloted MMA system. 


See Appendix Table A.1 for more information on the participating colleges in Minnesota, including 
graduation, enrollment, and retention rates, and averages for the entire system. 
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Wisconsin 


Wisconsin’s higher education system is relatively decentralized; colleges have autonomy to make 
placement decisions. The 16 two-year colleges in the Wisconsin Technical College System (WTCS) 
were in the early stages of considering multiple measures when MDRC began working with them." 


¢ Before the start of the pilot, all WTCS institutions used ACCUPLACER or the ACT’s COMPASS test, 
as well as accepting the ACT or similar tests, to place new students into courses, but they were 
exploring the feasibility of developing new systems. 


¢ Most WTCS institutions, including all those participating in this project, were piloting corequi- 
site courses during the 2016-2017 and 2017-2018 school years. The most widespread example was 
the use of the nationally recognized Accelerated Learning Program model in English, which had 
been implemented in several WTCS schools. 


These existing aspects of developmental education placement remained in effect during the pilot. 


See Appendix Table A.2 for more information on the participating colleges in Wisconsin, including 
graduation, enrollment, and retention rates, and averages for the entire system. 


1. Wisconsin has two college systems: the Wisconsin Technical College System and the University of Wisconsin 
System, whose two-year community colleges are referred to as the University of Wisconsin Colleges (UWC). Many 
of the UWC community colleges had already implemented some form of multiple measures assessment into their 
registration processes as of 2015. Only WTCS colleges participated in the Great Lakes pilot. 
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APPENDIX 


MMA Approaches at Ten Minnesota 
and Wisconsin Colleges 


APPENDIX TABLE B.1 


MMA Approaches at Colleges in the Great Lakes 
Multiple Measures Assessment Project 


NONCOGNITIVE COLLEGE-READY HIGH 


COLLEGE NAME STATE MMA APPROACH AND ORDER OF STEPS? ASSESSMENT SCHOOL GPA LEVEL? 
Anoka Ramsey MN 1. Exemptions (AP/IB, ACT, SAT, MCA LASSI 3.0 GPA 
scores) 
2. ACCUPLACER 
3. GPA and LASSI 
Century MN 1. Exemptions (ACT, SAT, MCA scores) LASSI 3.0 GPA 
2. ACCUPLACER 
3. GPA and/or LASSI 
Fox Valley WI 1. Exemption (ACT score) Grit Scale Not applicable 
2. ACCUPLACER 
3. Exemptions from further testing based 
on cut scores or program prep courses 
4. GPA + Grit + background questions 
added to ACCUPLACER score 
Lakeshore WI 1. Exemptions (GPA, ACT, GED scores; ACT Engage 2.8 GPA 
AP courses; certain programs exempt) 
2. ACCUPLACER 
3. GPA and/or ACT Engage 
Madison WI 1. Exemption (ACT score) ACT Engage 2.6 GPA 
2. ACCUPLACER (optional) 
3. GPA and ACT Engage 
Minneapolis MN 1. Exemptions (ACT, SAT, MCA scores) LASSI (optional) > 2.3 GPA for English, 
Community and 2. GPA or GPA and ACCUPLACER > 3.1 GPA for math, 
Technical > 2.4 GPA for reading 
Moraine Park WI 1. GPA and ACT Engage ACT Engage 3.25 GPA 
2. ACCUPLACER and ACT Engage 
Normandale MN 1. Exemptions (AP, ACT, SAT, MCA scores) LASSI 2.5-2.7 GPA for 
2. ACCUPLACER math, 2.1-2.3 GPA for 
3. GPA or GPA and LASSI English and reading 
North Hennepin MN 1. Exemptions (ACT, SAT, MCA scores) LASSI 3.0 GPA 
2. ACCUPLACER 
3. GPA and/or LASSI 
Northeast WI 1. Exemption (GPA) Grit Scale 2.6 GPA 
Wisconsin 2. ACT + Grit or ACCUPLACER + Grit 


(automated into ACCUPLACER) 


NOTES: The table shows preliminary decisions that were used in the pilot phase of the project in summer 2017. Some 
approaches have since been adjusted. 

aAP = Advanced Placement; IB = International Baccalaureate; GED = General Educational Development. Test scores that 
may result in exemptions include the ACT, the SAT, and the Minnesota Comprehensive Assessment (MCA). 

PAIl high school grade point averages (GPAs) are based on a 4.0 scale. 
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ABOUT MDRC 


MDRC IS A NONPROFIT, NONPARTISAN SOCIAL AND EDU- 
CATION POLICY RESEARCH ORGANIZATION DEDICATED TO 
learning what works to improve the well-being of low-income 
people. Through its research and the active communication of its 
findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 


Founded in 1974 and located in New York; Oakland, California; 
Washington, DC; and Los Angeles, MDRC is best known for 
mounting rigorous, large-scale, real-world tests of new and ex- 
isting policies and programs. Its projects are a mix of demon- 
strations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. 
MDRC’s staff members bring an unusual combination of research 
and organizational experience to their work, providing expertise 
on the latest in qualitative and quantitative methods and on pro- 
gram design, development, implementation, and management. 
MDRC seeks to learn not just whether a program is effective but 
also how and why the program’s effects occur. In addition, it tries 
to place each project’s findings in the broader context of related 
research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, 
and best practices are shared with a broad audience in the policy 
and practitioner community as well as with the general public and 
the media. 


Over the years, MDRC has brought its unique approach to an 
ever-growing range of policy areas and target populations. 
Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, 
employment programs for ex-prisoners, and programs to help 
low-income students succeed in college. MDRC’s projects are 
organized into five areas: 


e Promoting Family Well-Being and Children’s Development 
e Improving Public Education 

e Raising Academic Achievement and Persistence in College 
e Supporting Low-Wage Workers and Communities 

e Overcoming Barriers to Employment 


Working in almost every state, all of the nation’s largest cities, 
and Canada and the United Kingdom, MDRC conducts its proj- 
ects in partnership with national, state, and local governments, 
public school systems, community organizations, and numerous 
private philanthropies. 


